NPS-SM-95-007 


NAVAL  POSTGRADUATE  SCHOOL 
Monterey,  California 


ANALYSIS  AND  REDESIGN  OF  NAVY 
RETENTION/SEPARATION 
QUESTIONNAIRE 

by 

Alice  Crawford 
Julie  Dougherty 

October  1995 


Approved  for  public  release;  distribution  is  unlimited. 

Prepared  for:  Chief  of  Naval  Personnel  (Pers  232) 
2  Navy  Annex 

Washington,  DC  20370-5232 


19951103  066 


JOTIC  QOiijUATX  k 


NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CA  93943-5000 


Richard  E.  Elster 
Provost 


Marcia  J.  Evans,  RADM,  USN 
Superintendent 


This  report  was  prepared  for  and  funded  by  the  Chief  of  Naval  Personnel . 
Reproduction  of  all  or  part  of  this  report  is  authorized.  This  report  was  prepared  by: 

Department  of  Systems  Management 


JuUl^  Dougherty, 

Dq^artment  of  Systems  Management 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMBNo  0704-0188 

PubUc  reporting  burden  for  this  coUecSon  of  information  is  estimated  to  average  1  hour  per  response,  mduding  the  time  for  reviewmg  instructions,  searching  existing  data  sources, 
gathering  and  mainUining  the  dato  needed,  and  completing  and  reviewing  the  coUection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  as|^t  of  this 
coUection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  information  Operations  and  Reports,  1215  Jefferson  Davis 
Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 

1.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

October  1995  Technical  Report  4/95  to  9/95 

4.  TITLE  AND  SUBTITLE 

Analysis  and  Redesign  of  Navy  Retention/Separation  Questionnaire 

5.  FUNDING 

6.  AUTHOR(S) 

AliceCrawford 

Julie  Dougherty 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Department  of  Systems  Management 

Naval  Postgraduate  School 

555  Dyer  Rd. 

Monterey,  CA  93943-5003 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

NPS-SM-95-007 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Chief  of  Naval  Personnel  (Pers  232) 

2  Navy  Annex 

Washington,  DC  20370-5232 

to.  SPONSORING/MONITORING 

AGENCY  REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

12b.  DISTRIBUTION  CODE 

13.  ABSTRACT  (Maximum  200  words.) 

The  Navy  Separation/Retention  survey  (NSRS)  is  administered  to  Naval  enlisted  personnel  at  the  point  of  reenlistment  or  separation, 
and  to  officers  following  a  permanent  change  of  station  or  prior  to  separation.  The  data  allow  manpower  planners  to  anticipate  the  factors 
that  influence  decisions  to  stay  in  or  leave  the  Navy.  The  Bureau  of  Naval  Personnel  (Pers  23)  asked  the  Naval  Postgraduate  School, 
Systems  Management  Department  to  analyze  and  revise  the  NSRS  to  eliminate  some  problems  that  had  been  identified.  This  document 
describes  the  procedures  used  to  revise  the  NSRS,  the  new  survey  items  and  scales,  and  recommendations  for  follow-on  work  to  maintain 
a  quality  survey  for  the  Navy. 

14.  SUBJECT  TERMS 

Exit  surveys.  Navy  Retention 

15.  NUMBER  OF 

PAGES 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIUCA 1  lON 

OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

unclassified  unclassified  unclassified 

vicxT -icAA  Ai  -)0A  «oAn  standard  honu  2v 

20.  LIMITATION  OF 
ABSTRACT 

8  (Rev. 2-89) 

Prescribed  by  ANSI  Std  239-18 


ANALYSIS  AND  REDESIGN  OP  NAVY 
RETENTION/ SEPARATION  QUESTIONNAIRE 


By 
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Julie  Dougherty 


Naval  Postgraduate  School 
Systems  Management  Department 
Monterey,  CA  93943 


EXECUTIVE  SUMMARY 


The  Navy  Separation/Retention  Survey  (NSRS)  is  administered  to 
Naval  enlisted  personnel  at  the  point  of  reenlistment  or 
separation,  and  to  officers  following  a  permanent  change  of  station 
or  prior  to  separation.  The  data  allow  manpower  planners  to 
anticipate  the  factors  that  influence  decisions  to  stay  in  or  leave 
the  Navy. 

Several  problems  with  the  current  survey  have  been  identified. 
Administrative  procedures  are  inconsistent  and  participation,  which 
is  voluntary,  is  low.  These  two  factors  combined  indicate  that 
respondents  may  not  represent  the  population.  Additionally,  there 
are  considerable  overlap  and  ambiguity  among  the  questionnaire 
items,  making  analysis  and  policy  decisions  difficult. 

The  Naval  Postgraduate  School  (NPS),  Systems  Management 
Department  proposed  to  analyze  and  revise  the  current  survey  in 
response  to  a  request  initiated  by  the  Bureau  of  Naval  Personnel 
(Pers  23)  .  The  two  organizations  agreed  that  the  focus  of  the 
effort  would  be  qn  the  redesign  of  the  questionnaire  items  to 
create  a  survey  no  longer  than  one  page.  This  would  allow  for 
continuity  for  analysis  purposes  and  would  not  require  additional 
time  for  Naval  personnel  to  complete  the  questionnaire. 

A  factor  analysis  was  used  to  identify  ambiguous  items  in  the 
current  survey.  Since  the  NSRS  had  not  been  revised  since  1990, 
the  Nominal  Group  Technique  was  used  to  elicit  new  issues  that  are 
satisfying  or  dissatisfying  to  Naval  personnel.  Nominal  Group 
interviews  were  conducted  at  the  Naval  Postgraduate  School  and 
onboard  the  USS  Carl  Vinson. 

A  list  of  136  potential  NSRS  items  was  forwarded  to  the  CNO 
Career  Information  Team  (Pers  232)  for  input  on  reducing  the  list 
to  a  number  that  would  not  exceed  the  one-page  requirement.  The 
Career  Information  Team,  at  the  direction  of  the  Under  Secretary  of 
the  Navy,  is  conducting  focus  groups  with  fleet  personnel  who  are 
voluntarily  separating  before  retirement.  As  such,  the  team 
members  are  familiar  with  current  issues  and  concerns  within  the 
fleet.  Both  positive  and  negative  issues  most  frequently  mentioned 
to  NPS  researchers  and  the  Career  Information  Team  were  kept,  items 
available  from  other  Navy  surveys  were  eliminated  as  feasible,  and 
the  final  list  was  derived  by  an  assessment  by  Pers  232  of  the 
issues  most  relevant  to  the  current  needs  of  Navy  manpower  planners 
and  policy  makers.  The  revised  NSRS  includes  35  items  and  two 
measurement  scales. 

The  following  recommendations  are  made: 

1.  Review  the  questionnaire  periodically. 

2.  Collect  regular  feedback  from  data  users. 

3.  Conduct  a  factor  analysis  on  the  revised  NSRS. 

4.  Develop  and  implement  a  procedure  for  consistent 
administration. 

5.  Consider  implementation  of  a  computerized  NSRS. 


INTRODUCTION 


This  report  describes  the  revision  of  the  Navy 
Separation/Retention  Survey  (NSRS),  which  was  conducted  in  the 
Sprinq  of  1995.  The  survey  provides  the  data  needed  by  Navy 
manpower  planners  to  anticipate  the  factors  that  influence 
decisions  to  stay  in  or  leave  the  Navy  and  to  improve  and  develop 
personnel  related  policies  and  initiatives.  There  has  been 
increasing  attention  to  this  survey  because  of  the  drawdown  and 
subsequent  need  for  retaining  the  proper  mix  of  skilled,  quality 
people.  Both  the  Marine  Corps  and  the  Army  are  also  revising  their 
separation  surveys  during  this  period. 


BACKGROUND 

The  process  of  exit  interviewing  and  surveying  (EIS)  was 
developed  as  a  means  of  gathering  data  from  a  person  whose 
employment  with  an  organization  had  been  voluntarily  or 
involuntarily  terminated  (Goodale,  1982) .  EIS  usually  addresses 
issues  such  as  pay,  benefits,  working  conditions,  opportunities  for 
advancement,  the  quality  and  quantity  of  the  workload,  and  the 
rapport  between  coworkers  and  supervisors.  (More  background  on 
exit  interviewing  and  surveying  is  provided  in  Appendix  A.) 

OPNAVINST  1040. 8D  authorizes  the  Navy  Separation/Retention 
Questionnaire  for  data  collection  and  policy  analyses.  While  the 
completion  of  the  questionnaire  is  voluntary,  the  instruction 
directs  commanding  officers  to  ensure  maximum  participation  from 
enlisted  personnel  at  the  point  of  extension  for  more  than  2 
months,  reenlistment  or  separation,  and  from  officers  following 
execution  of  Permanent  Change  of  Station  (PCS)  orders  or  upon 
separation.  Parent  commands  are  directed  to  submit  questionnaires 
with  information  on  the  front  page  for  personnel  declining  to 
participate  voluntarily. 

Besides  providing  basic  demographic  information,  respondents 
are  asked  to  rate  satisfaction  with  45  aspects  of  Navy  life  (Part 
A),  and  to  identify  the  one  factor  of  the  45  that  makes  (or  made) 
the  respondent  think  about  leaving  the  Navy  the  most  (Part  B)  .  The 
data  can  be  used  in  the  formulation  of  future  policy  and  lend 
support  for  current  initiatives  that  affect  Naval  service  members. 

Several  problems  with,  the  current  survey  have  been  noted  and  were 
documented  by  the  Center  for  Naval  Analyses  (Sharma,  1994) .  First, 
participation  has  been  low.  This  was  also  addressed  in  a  message 


3 


from  the  Chief  of  Naval  Personnel  noting  that  of  69,865  enlisted 
separations,  in  FY  94,  only  9,511  (13.6%)  filled  out  the 
questionnaire  and  that  of  the  13,227  (non-student  status)  _ of freer 
PCS  transfers  in  FY  94,  only  1,130  (8.5%)  officers  participated 
(CNO/Nl  WASH  DC  NAVADMIN  230/94) .  CNA  addressed  concerns  of  both 
nonresponse  and  selection  bias  that  potentially  exist.  There  is 
great  concern  that  those  who  respond  may  not  represent  the  entire 
group  and,  thus  the  data  collected  may  not  be  indicative  of  trends 
throughout  the  fleet. 

Second,  it  was  noted  by  the  Center  for  Naval  Analyses  (CNA) 
that  there  was  considerable  overlap  among  the  45  items  in  Part  A. 
Some  items  seem  to  address  the  same  issues  in  different  ways,  and 
other  items  appear  so  similar  that  respondents  may  have  difficulty 
distinguishing  one  from  the  other.  Further,  there  are  no  subject- 
related  divisions  among  the  questions  allowing  potential 
measurement  error  to  arise  from  the  respondent  taking  less  care  in 
answering  each  question. 

Additionally,  CNA  pointed  out  that  the  45  choices  available  to 
choose  from  for  Part  B  are  excessive.  In  fact,  the  data  they 
analyzed  from  1990-1992  showed  small  response  frequencies  for  this 
item.  This  presents  another  type  of  measurement  error  since  the 
reason  for  nonresponse  is  not  known. 

Following  the  CNA  study,  a  memorandum  was  written  to  the 
SUPERS  Research  Management  Advisor  by  the  Assistant  Chief  of  Naval 
Personnel  for  Military  Personnel  Policy  and  Career  Progression  that 
requested  a  study  to  analyze  the  administration  and  results  of  the 
Navy  Retention/Separation  Questionnaire  (Konetzni,  1995)  . 

In  response  to  the  concern  shown  by  the  Bureau  of  Naval 
Personnel  (Pers  23)  with  the  problems  in  the  current  survey,  the 
Naval  Postgraduate  School  (NPS),  Systems  Management  Department, 
submitted  a  proposal  to  analyze  and  revise  the  survey  during  the 
period  31  March  1995  to  30  September  1995.  A  meeting  .was  held  in 
April  1995  in  which  representatives  from  NPS  and  Pers  23  agreed 
that  the  proposed  effort  would  be  restricted  to  revision  of  the 
survey  items  and  that  the  administrative  problems  of  the  survey 
would  be  worked  out  at  some  time  in  the  future.  It  was  also  agreed 
that  the  new  survey  could  be  no  longer  than  the  old  survey,  which 
is  a  single  page  with  one  side  for  demographic  data  and  the  other 
for  the  actual  survey  questions.  This  decision  was  made  due  to 
both  time  and  resource  constraints  and  a  desire  to  keep  the  data 
categories  relatively  consistent  with  those  that  have  been  used 
since  the  survey  was  developed.  Pers  23  should  be  able  to  continue 
to  monitor  trends  that  have  already  been  identified  as  indicators 
of  satisfaction  or  dissatisfaction  with  the  Navy. 
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APPROACH 


Factor  Analysis 

A  principal  components  factor  analysis  of  the  45  items  on  the 
current  Navy  Retention/Exit  Questionnaire  revealed  seven  factors 
accounting  for  54.7  percent  of  the  variance,  all  having  eigen 
values  greater  than  1.0.  In  general,  items  tend  to  load 
significantly  on  one  factor.  However,  four  items  loaded  on 
different  factors  probably  because  the  wording  was  vague  leaving 
the  item  open  to  multiple  interpretations.  One  item.  Amount  of 
enjoyment  from  my  job"  was  eliminated  because  it  was  too  general. 
The  other  three  items  were  reworded  to  be  consistent  with  the 
factor  they  were  attempting  to  measure. 

Survey  Item  Revision 

While  CNA  had  made  it  clear  that  the  survey  items  needed  to  be 
revised  to  eliminate  redundancy,  etc.,  it  seemed  equally  clear  to 
the  present  researchers,  given  that  the  survey  had  not  been  revised 
since  1990,  that  the  items  might  be  out  of  date.  It  seemed  likely 
that  there  were  items  that  could  be  eliminated  because  they  were  no 
longer  relevant,  or  because  they  are  addressed  on  other  Navy 
surveys.  Further,  in  light  of  significant  changes  in  the  Navy  in 
recent  years,  it  also  seemed  likely  that  there  would  be  a  need  to 
add  new  items  to  reflect  the  current  Navy  environment. 

To  determine  the  current  issues  of  most  importance  to  Navy 
people,  a  short  survey  was  administered  to  52  officers  at  NFS  who 
were  students  in  a  management  course  that  is  a  requirement  for  all 
of  the  Systems  Management  curricula.  The  class  included  students 
of  most  major  Navy  designators,  and  were  primarily  Lieutenants  with 
some  Lieutenant  Commanders.  It  is  not  assumed  that  this  group 
necessarily  represents  the  Navy  population.  Rather,  this  first 
step  was  undertaken  to  (1)  give  the  researchers  an  idea  of  the 
range  of  issues  that  should  be  expected,  and  (2)  to  use  later  to 
compare  with  the  main  body  of  data  collected  from  group  interviews. 
The  NFS  student  survey  asked  for  rank,  designator,  curriculum, 
intentions  to  stay  in  the  Navy,  what  they  like  most  about  being  in 
the  Navy,  and  what  they  like  least  about  being  in  the  Navy. 


The  NFS  student  data  revealed  a  wide  range  of  issues  of 
concern,  more  than  could  be  constrained  to  a  one~page  survey.  On 
the  other  hand,  the  data  showed  a  remarkable  consensus  about  what 
people  like  about  being  in  the  Navy. 

Subsequently,  group  interviews  were  conducted  with  shore-based 
personnel  from  NFS  and  personnel  on  board  the  USS  Carl  Vinson.  The 
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NFS  groups  included  21  enlisted  personnel  divided  into  four  groups 
called  "First  Termers"  who  are  in  their  initial  tour  of 
"Careerists"  who  have  more  than  six  years  of  service  and  intend  o 
make  the  Navy  a  career,  "Retirees"  who  were  already  scheduled  for 
retirement,  and  "Separating  Personnel"  who  were  voluntarily 
separating.  The  same  four  groups  were  represented  in  the  people 
interviewed  from  the  USS  Carl  Vinson,  which  included  21  officers 
and  14  enlisted  members. 

Nominal  Group  Procedures 

The  group  interview  procedure  used  was  the  Nominal  Group 
Technique,  which  has  been  used  by  Giacalone  (1993)  for  collecting 
data  relevant  to  exit  surveys.  The  procedure  involved  the 
following  steps: 

1 .  Groups  were  assembled,  welcomed,  and  briefed  on  the 
purpose  of  the  meeting. 

2.  Each  individual  was  asked  to  fill  out  a  form  that  asked 
for  demographic  data  (shown  in  Appendix  B) .  Group  members  were 
assured  that  no  attempt  would  be  made  to  connect  their  names  with 
the  data  to  be  collected  during  the  meeting. 

3.  Individuals  were  asked  to  fill  out  a  second  form  (Appendix 
B)  that  asked  them  to  list  the  things  they  like  most  and  least 
about  being  in  the  Navy.  They  were  asked  to  think  about  their 
total  Navy  experience,  not  to  concentrate  just  on  the  present 
command  or  most  recent  experience. 

4  Group  members  were  asked  to  engage  in  a  discussion  of  the 
items  they  had  listed  that  they  liked  least  about  being  in  the  Navy 
until  all  items  had  been  mentioned  and  the  entire  group  was  clear 
on  the  meaning  of  each  item.  Each  item  was  written  on  easel  paper, 
numbered  in  order  of  mention,  and  taped  to  the  wall  of  the  meeting 
room.  The  discussions  lasted  about  50-60  minutes. 

5.  Individuals  were  asked  to  fill  out  another  form  (Appendix 
B)  in  which  they  rated  each  numbered  item  as  either  "0"  (does  not 
apply)  or  on  a  scale  ranging  from  "1,2,3"  (not  very  important)  to 
"4, 5, 6, 7"  (moderately  important),  to  "8,9,10"  (very  important). 

6.  Steps  4  and  5  were  repeated  for  the  items  that  people 
listed  as  those  things  they  liked  most  about  being  in  the  Navy. 

7 .  Final  questions  were  answered  and  the  groups  were  thanked 
and  dismissed. 
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Data  Analysis 


The  group  interview  data  were  aggregated  to  pool  all  responses 
from  all  groups.  Although  all  groups  did  not  list  the  exact  same 
items  as  satisfiers  or  dissatisf iers,  there  were  enough 
similarities  among  the  groups  that  the  researchers  were  able  to 
compare  and  combine  results  into  aggregate  data.  Responses  that 
had  been  rated  as  "8,"  "9,"  or  ”10,”  (very  important)  by  more  than 
half  of  the  respondents  were  kept.  This  list  of  responses  was 
compared  to  those  obtained  from  the  NFS  students  and  found  to  be 
very  similar.  Thus,  the  response  set  seemed  to  be  comprehensive 
and  there  appeared  to  be  nothing  potentially  valuable  that  had  been 
missed.  This  response  set  included  136  items.  The  next  task  was  to 
reduce  the  response  set  to  a  number  that  could  be  represented  on  a 
one  page  survey. 

The  survey  items  were  forwarded  to  Pers  232  for  their 
assistance  in  the  evaluation.  Pers  232  is  part  of  the  Career 
Information  Team,-  which  is  tasked  with  providing  the  latest 
information  on  personnel  policy  issues  to  the  fleet  and  reporting 
fleet  concerns  back  to  senior  Naval  leaders  in  Washington,  DC.  In 
this  capacity,  the  team  members  are  familiar  with  all  of  the 
current  concerns  of  fleet  personnel,  and  aware  of  which  of  these 
seem  the  most  important  at  any  given  time.  Additionally,  the 
Career  Information  Team,  at  the  direction  of  the  CNO,  is  in  the 
process  of  conducting  focus  groups  with  fleet  personnel  who  are 
voluntarily  separating  before  retirement  in  order  to  learn  what 
influences- their  decisions.  Therefore,  it  was  decided  that  the 
team  members  were  the  most  qualified  to  help  reduce  the  list  of 
potential  items  to  a  comprehensive,  yet  manageable  number  of  survey 
items . 

Pers  232  recommended  a  list  of  31  items  to  NPS  researchers, 
which  were  then  compared  to  a  list  of  39  items  independently 
derived  by  NPS.  After  several  discussions  between  the  two 
organizations,  a  final  list  of  35  items  was  agreed  upon.  The 
criteria  for  inclusion  in  the  final  list  were: 

1.  Items  frequently  mentioned  as  negative  factors  (i.e., 
factors  that  could  cause  people  to  leave  the  Navy)  in  NPS 
nominal  group  interviews  and  Career  Information  Team  focus 
groups.  Many  of  these  items  are  currently  under  review  by 
policy  makers,  for  example  the  military  retirement  system  may 
be  changed.  Keeping  such  items  in  the  survey  will  be 
important  so  that  data  trends  can  be  monitored. 

2.  Items  frequently  mentioned  as  reasons  for  staying  in 
the  Navy.  Considering  all  possible  uses  for  the  questionnaire 
data,  it  may  become  increasingly  important  to  communicate  and 
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build  on  the  factors  perceived  as  motivators. 

3.  Items  used  on  other  surveys  were  not  included. 

4.  Items  considered  by  Pers  232  to  be  most  relevant  to 
current  needs  of  Navy  manpower  planners  and  policy  makers. 


RESULTS 


Questionnaire  Items 

The  35  items  for  the  revised  questionnaire  are: 

1 .  Amount  of  family  separation 

2.  Number  of  inspections 

3.  Challenge  of  work 

4.  Length  of  work  days 

5.  Amount  of  personnel  to  do  jobs 

6.  Amount  of  responsibility 

7.  Competence  of  coworkers 

8.  Opportunities  to  serve  in  leadership  roles 

9.  Type  of  assignments 

10.  Participation  in  assignment  decisions 

11.  Impact  of  PCS  on  spouse's  career 

12.  Travel  opportunities 

13.  Advancement  opportunities 

14.  Job  security 

15.  Friendships  made  in  the  Navy 

16.  Pay  compared  with  local  living  costs 

17.  Pay  compared  with  other  civilian  jobs 

18.  Cost  of  living  allowances 

19.  Personal  access  to  medical  and  dental  care 

20.  Quality  of  personal  medical  and  dental  care 

21.  Access  to  medical  and  dental  care  for  family  members 

22 .  Quality  of  medical  and  dental  care  for  family  members 

23.  Access  to  military  housing 

24.  Quality  of  military  housing 

25.  Possible  reduction  in  retirement  benefits 

26.  Paid  vacation  time 

27.  Access  to  job-related  training 

28.  College  education  programs 

29.  Employee  assistance  programs  (financial,  etc.) 

30.  Consistent  enforcement  of  rules/regulations 

31.  Leadership  ability  of  immediate  supervisor 

32.  Recognition  from  leaders 

33.  Support  up  the  chain  of  command 

34.  Accuracy  and  consistency  of  performance  evaluations 

35.  Quality  of  career  information 
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Scales 


Two  rating  scales  were  created.  The  first  scale  addresses  the 
importance  of  the  issue  to  the  individual  in  his/her  decision  to 
stay  in  or  leave  the  Navy: 

1.  Not  at  all  important 

2 .  Somewhat  important 

3 .  Very  important 

4.  Not  applicable 

The  second  scale  measures  whether  the  individual  is  (1)  satisfied 
or  (2)  dissatisfied  with  the  issue.  Both  scales  are  essential  for 
effective  use  of  the  questionnaire  data.  While  all  of  the  items 
are  important  to  the  respondents  in  some  sense,  there  will  be 
differences  among  subgroups  of  individuals  (e.g.,  officers  vs. 
enlisted,  stayers  vs.  leavers,  etc.),  and  there  will  be  changes 
over  time.  Using  the  two  scales  will  allow  users  to  focus  on  the 
two  factors  for  any  given  data  extraction.  That  is,  which  subsets 
of  items  are  seen  as  relatively  more  important  to  the  respondents, 
and  the  level  of  satisfaction  that  respondents  feel  regarding  those 
issues. 

Part  (B)  of  the  original  survey,  which  asked  respondents  to 
choose  the  one  item  that  makes  (or  made)  them  think  about  leaving 
the  Navy  the  most,  is  no  longer  required.  The  two  measurement 
scales  should  give  enough  information  about  the  respondents' 
decisions  without  that  single  question.  Additionally,  although  the 
list  of  items  that  the  respondent  has  to  choose  from  has  been 
reduced  from  45  to  35,  choosing  a  single  one  would  still  be 
difficult  for  most  respondents. 
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RECOMMENDATIONS 


1,  Review  the  questionnaire  periodically.  An  update  or 
revision  "of  the  questionnaire  items  must  be  considered 
periodically.  Although  such  an  update/revision  need  not  be 
extensive,  the  changing  environment  in  the  Navy,  as  well  as 
external  technological  and  social  changes  that  impact  Navy  life, 
may  result  in  the  addition  of  new  items  and/or  the  revision  or 
deletion  of  existing  items.  One  way  to  monitor  the  need  for  change 
is  to  include  additional  space  for  respondents  to  indicate  if  there 
are  other  items  not  mentioned  that  are  important  to  their  decision 
to  stay  or  leave  the  Navy  and  whether  they  are  satisfied  or 
dissatisfied  with  that  particular  issue. 

2,  Get  regular  feedback  from  data  users.  While  the  current 
items  may  satisfy  Navy  needs  now,  it  is  important  to  realize  that 
in  the  future,  the  needs  of  those  who  use  the  data  may  change. 
Additionally,  new  data  users  may  develop  over  time.  Such  changes 
could  result  in  a  need  to  change  the  questionnaire  items. 

3 ,  Conduct  a  factor  analysis  on  the  data  collected  from  the 
new  questionnaire.  Once  the  revised  questionnaire  is  in  place,  it 
would  be  appropriate  to  analyze  the  new  data  collected  and 
determine  whether  the  factors  derived  from  the  analysis  fit  Navy 
expectations  and  whether  all  items  are  loading  cleanly  on  one 
factor  (i.e.,  are  unambiguous). 

4 ,  Develop  and  implement  a  procedure  for  consistent 
administration  of  the  questionnaire.  There  is  a  need  for  the 
development  of  a  procedure  for  administering  the  instrument,  as 
well  as  closer  monitoring  of  the  consistency  with  which  the 
procedure  is  followed.  It  is  recommended  that  the  policy  of  the 
Navy  by  analyzed  and  critiqued  in  order  to  determine  an  effective 
way  to  proceed  with  policy  development.  Active  participation  of 
transition  sites  via  interviews,  phone  calls,  and  surveys  should  be 
undertaken  to  determine  how  the  procedures  and  policy  might  best  be 
formulated  to  increase  the  quality  and  amount  of  data  as  well  as  to 
minimize  interference  in  the  varying  transition  points.  A  formal 
procedure,  based  on  the  research,  should  be  developed  and  sent  to 
responsible  personnel  at  the  transition  points.  Furthermore,  a 
clarification  of  the  meaning  of  "voluntary  participation"  may  be 
needed. 

5,  Consider  implementing  a  computerized  instrument.  The 
Navy  could  benefit  by  changing  from  a  paper  and  pencil  measure  to 
a  computerized  format.  This  format  would  have  many  beneficial 
outcomes.  Particularly,  computerizing  the  instrument  would  make  it 
more  closely  resemble  an  exit  interview  (without  the  expense  of 
individual  interviewers  or  the  interpersonal,  nonverbal  cues  that 


10 


interviewers  may  reveal)/  and  would  increase  the  efficacy  of  the 
instrument  for  many  reasons.  There  would  be  no  out~of~rang6  ot 
ji^uTtiple  responses/  there  would  be  automatic  data  entry/  missing 
data  could  be  disallowed/  and  survey  time  could  be  quiclcer.  More 
importantly/  it  would  allow  for  interview-like  follow  ups  (via 
branching  techniques)  and  would  be  more  easily  adaptable  for 
special  topics  and  quick  response  tasks,  thereby  allowing  the  Navy 
to  insert  new  items  and/or  supplements  without  the  concern  for 
printing  and  the  sunk  costs  of  surveys  already  printed.  Such  a 
computerized  instrument  would  also  allow  for  much  quicker  reporting 
time  to  sponsors. 
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APPENDIX  A 


EXIT  INTERVIEWING  AND  SURVEYING 


EXIT  INTERVIEWING  AND  SURVEYING 


The  process  of  exit  interviewing  and  surveying  (EIS)  was 
developed  as  a  means  of  gathering  data  from  a  person  whose 
employment  with  an  organization  had  been  voluntarily  or 
involuntarily  terminated  (Goodale/  1982) .  EIS  usually  concerns 
issues  such  as  pay/  benefits/  working  conditions/  opportunities  for 
advancement/  the  quality  and  quantity  of  the  workload/  and  the 
rapport  between  coworkers  and  supervisors.  The  EIS  process 
provides  information  for  three  distinct  purposes:  diagnosis  and 
strategy  (turnover/  training  and  development  needS/  and  the 
creation  of  planning  goals) ,  public  relations  (the  transition  from 
an  employee  status  to  a  customer/  and  personal  catharsis  (to  bring 
about  an  amicable  parting  between  employee  and  company) . 

The  premise  behind  EIS,  that  parting  employees  will  give 
complete  and  accurate  information/  is  quite  idealistic.  EIS  has 
often  been  criticized  for  two  reasons.  First/  administrative 
critiques  of  the  process  have  argued  that  EIS  is  a  symbolic  gesture 
(because  no  use  is  made  of  the  information  obtained)  /  and  that  the 
process  itself  is  done  hurriedly  on  the  employee's  last  day  of  work 
(Woods  &  Mccauley/  1987) ,  making  the  data  questionable. 
Additionally/  when  dissatisfaction  is  reported  by  the  exiting 
employee/  the  feedback  to  management  is  lacking  because  the  EIS 
administrator  (for  different  reasons)  often  fails  to  report  it 
(Hinrich/  1975) . 

Secondly/  methodological  critiques/  which  have  attended  to  the 
ifiore  serious  issues  of  reliability  and  validity/  have  argued  that 
the  underlying  techniques  used  in  many  EIS  processes  are 
fundamentally  problematic.  For  example/  in  the  exit  interview/ 
because  the  exit  interviewer  is  not  always  completely  objective 
(HinrichS/  1971)/  interviewee  defensiveness  may  result.  Exiting 
employees  may  therefore  mitigate  the  severity  of  the  problem  areas, 
overstate  the  positive  nature  of  generally  satisfactory  issues,  or 
provide  themselves  with  an  opportunity  to  retaliate  against  the 
company  via  responses  that  are  directly  the  opposite  of  felt 
concerns.  Additionally,  some  studies  (Hinrichs,  1971;  Zaradona  & 
Camuso,  1985)  have  shown  potential  inconsistencies  over  time  in 
explanations  for  leaving  provided  by  separating  personnel. 

Although  there  are  legitimate  concerns  with  EIS  processes, 
nowhere  in  the  literature  is  the  EIS  process  condemned  as  an 
ineffective  process,  per  se;  the  administrative  and  methodological 
concerns  are  curable  ones.  Many  EIS  problems  are  caused  by 
comparatively  simple  issues  of  poor  planning,  lack  of  insight, 
ignorance  of  methodology,  and  a  host  of  other  mistakes  associated 
with  a  "quick  fix"  mentality.  It  is  important  to  note  that  even 
with  these  problems,  the  EIS  process  is  not  different  from  other 
human  resource  management  processes  and  most  be  adjusted  to  deal 


with  the  administrative  and  methodological  problems  inherent  to  the 
particular  process. 


APPENDIX  B 


FORMS  USED  FOR  GROUP  INTERVIEWS 


IN  ORDER  TO  ASSIST  US  IN  EVALUATION  OF  THE  INFORMATION  RECEIVED  IN  THIS 
DISCUSSION  GROUP,  REQUEST  YOU  ANSWER  THE  FOLLOWING  QUESTIONS.  ALL 
YOUR  RESPONSES  TO  THESE  QUESTIONS  AND  TO  THOSE  ASKED  AS  PART  OF  THE 
DISCUSSION  WILL  REMAIN  COMPLETELY  ANONYMOUS. 

WHEN  DID  YOU  ENTER  THE  NAVY? _ _ _ 

WHEN  IS  YOUR  EAOS? _ _ _ 

HOW  MANY  TIMES  HAVE  YOU  REENLISTED?  _ _ _ _ _ 

ARE  YOU  MARRIED? _  SINGLE? - 

NUMBER  OF  DEPENDENTS?  _ _ 

EDUCATION  LEVEL?  _ _ _ _ _ _ 


THANK  YOU  FOR  YOUR  ASSISTANCE.  WE  VALUE  YOUR  CONTRIBUTION  AND 
APPRECIATE  THE  TIME  AND  ENERGY  THAT  YOU  HAVE  DEVOTED  TO  THIS 
DISCUSSION.  IF  YOU  HAVE  ANY  QUESTIONS,  PLEASE  FEEL  FREE  TO  CONTACT  US 


Professor  Alice  Crawford 
LCDR  Julie  A.  Dougherty 


NPS  SEPARATION/RETENTION  QUESTIONNAIRE 


Please  list  the  things  you  like  the  LEAST  about  being  in  the 
Navy.  Try  to  think  of  anything  that  has  made  you  feel  dissatisfied 
with  your  job,  your  work  environment,  or  the  effects  of  the  Navy  on 
your  personal  life  both  here  and  at  any  previous  places  in  the 
Navy. 


♦ 

Now  list  the  things  you  like  MOST  about  being  in  the  Navy. 
Try  to  think  of  all  the  things  that  make  you  satisfied  with  your 
job,  your  work  environment,  or  the  effects  of  the  Navy  on  your 
personal  life.  Again,  consider  all  of  your  Navy  e5<periences  as  you 
answer  this  question. 


RATINGS  FOR  DISSATISFIERS 


We  would  like  you  to  rate  each  of  the  items  that  have  been 
submitted  by  you  and  the  other  group  members.  Using  the  scale 
shown  below,  please  assign  a  value  to  each  of  the  items  (0  through 
10)  that  describes  how  important  the  item  could  be  (or  was)  in 
making  you  decide  to  leave  the  Navy. 


Scale : 


0 


1,  2,  3, 


4,  5,  6,  7, 


8,  9,  10 


Doesn't 


Not  V«ry  Ini>ortaiit 


Modoratoly  Zoportant 


Very  lBf>ortaat 


Item#  Importance  Rating 


Item#  Importance  Rating 


1. 

2. 

3. 

4. 

5. 

6 . 
7. 

8  . 

9. 

10. 
11 . 
12. 
13. 
14  . 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 


31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 

41 . 

42. 

43. 

44. 

45. 

46. 

47. 
48  . 

49. 

50. 

51. 

52. 

53. 

54. 

55. 

56. 

57. 
58  . 

59. 

60. 


RATINGS  FOR  SATISFIERS 


We  would  like  you  to  rate  each  of  the  items  that  have  been 
submitted  by  you  and  the  other  group  members.  Using  the  scale 
shown  below,  please  assign  a  value  to  each  of  the  items  (0  through 
10)  that  describes  how  important  the  item  could  be  (or  was)  in 
making  you  decide  to  stay  in  the  Navy. 


Scale : 


0 


1,  2,  3, 


4,  5,  6,  7, 


8,  9,  10 


Doesn't  i^ly  Not  V«ry  Iiii>ortant 

Item  #  Importance  Rating 


ModuraltAly  Inportant 


V«ry  Ixqportant 


Item  #  Importance  Rating 


1. 

2. 

3. 

4. 

5. 

6 . 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 

21. 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 


31 . 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 

41. 

42. 

43. 

44. 

45. 

46. 

47. 

48. 

49. 

50. 

51. 

52. 

53. 

54. 

55. 

56. 

57. 

58. 

59. 

60. 
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